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identified patent application filed simultaneously herewith on August 30, 1999. 

IN THE SPECIFICATION: 

On page 1, line 1 of the specification, please add: 

-This application is a continuation of copending U.S. Patent Application Serial 
No. 08/689,678, filed on August 13, 1996, now U.S. Patent No. 5,944,530 and is 
incorporated by reference into this application.— 

IN THE CLAIMS: 



Sir: 



Please enter the following Preliminary Amendment for the above- 



Please delete claims 1 through 37; and add new Claims 38 through 43 as 

follows. 

38. A computer-aided-learning method for helping a student study materials on a 
subject comprising the step of: 

monitoring automatically more than once a concentration-sensitive behavior of the 
student through a non-intrusive sensor while the student should be studying materials on 
the subject, 

so that the monitored results can be analyzed to provide an indication on the 
student's concentration level, where monitoring more than once increases the accuracy of 
the indication; 

such that the behavior can be monitored with the sensor detached from the 
student. 

39. A computer-aided-learning method as recited in Claim 38 further comprising the 
step of reacting, by the computer, in view of the student's concentration level determined 
from the monitored results, so that the reaction can enhance the student's concentration 
level in studying materials on the subject. 

40. A computer-aided-learning method for helping a student study materials on a 
subject, the method comprising the steps of: 

monitoring a concentration-sensitive behavior of the student through a non- 
intrusive sensor while the student should be studying materials on the subject; and 

reacting, by the computer, in view of the student's concentration level determined 
from the monitored results, so that the reaction can enhance the student's concentration 
level in studying materials on the subject. 

41 . A computer-aided-learning method as recited in Claim 40 wherein the step of 
reacting includes the step of asking the student, a question. 



42. A computer-aided-learning system for helping a student study materials on a 
subject comprising: 

a non-intrusive sensor configured to monitor automatically more than once a 
concentration-sensitive behavior of the student while the student should be studying 
materials on the subject; 

so that the monitored results can be analyzed to provide an indication on the 
student's concentration level, where monitoring more than once increases the accuracy of 
the indication; 

such that the behavior can be monitored with the sensor detached from the 
student. 

43. A computer-aided-learning system for helping a student study materials on a 
subject comprising: 

a non-intrusive sensor configured to monitor a concentration-sensitive behavior of 
the student while the student should be studying materials on the subject; 

a controller configured to react, in view of the student's concentration level 
determined from the monitored results, so that the reaction can enhance the student's 
concentration level in studying materials on the subject. 

Dated: August 30, 1999 



Respectfully submitted, 




Peter P. Tong 
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Mountain View, CA 94040 
(650) 961-7789 



A LEARNING METHOD AND SYSTEM 
THAT CONSIDER A STUDENT'S CONCENTRATION LEVEL 



BACKGROUND OF THE INVENTION 

5 

The present invention relates generally to education, and more particularly to 
considering a student's concentration level in teaching the student through a computer. 

Both at home and in schools, the computer is gradually becoming a major medium 
for education. There are many different reasons for this trend. One is the tremendous 

10 reduction in the price of a computer, causing it to permeate into almost every household. 
Though the price of a computer has been dropping, its computation and memory capacity 
have increased many folds, leading to computer programs with significantly more 
intelligence and improved user-friendliness. Another reason is that a computer-aided- 
education system can be very personalized; it can be tailored to the strengths and 

15 weaknesses of individual students. This is very hard to achieve in today's educational 
environment, in part due to the increase in the students-per-instructor ratio. 

One weakness of a computer-aided-educational system is that it typically ignores a 
student's concentration level. In contrast, a good instructor teaches according to the 
student's attention span. She can sense whether or not the student is paying attention. 

20 Normally, she attributes her sensitivity to her "intuition," based on years of her teaching 
experience. One "intuition" is that when the student's pupils start to dilate, the student has 
lost focus. Another "intuition" is that when the student frowns, he is concentrating. Such 
"intuition" is very usefal in teaching, A good instructor constantly observes such 
concentration-sensitive behavior, and dynamically adjusts her teaching materials and style 

25 accordingly. If most of the students are looking elsewhere instead of at her for a certain 
period of time, the instructor might stop teaching and tell a joke. This helps the students 
re-focus back at the instructor. If a student is drooping, the instructor might directly ask 
her a question to "wake her up." Such important "intuition" is missing in today's 
computer-aided-educational systems. 
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It should be apparent from the foregoing that there is a need for a computer-aided- 
educational system and method that consider the student's concentration level, while the 
student is working on the study-materials. 
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SUMMARY OF THE INVENTION 



The present invention provides a computer-aided-educational system and method 
that automatically consider a student's concentration-sensitive behavior while the student 

5 is working on the study materials. 

In one embodiment, the present invention includes a presenter, a non-intrusive 
sensor, a controller and an indicator. The presenter presents study materials on a subject 
to the student; the non-intrusive sensor automatically monitors more than once the 
student's concentration-sensitive behavior while the student is working on the materials; 

10 the controller analyzes the student's concentration-sensitive behavior based on one or 
more rules; and the indicator provides an indication on the student's concentration level 
based on the analysis. In another embodiment, the present invention reacts according to 
the indication. 

There are a number of examples of the concentration-sensitive behavior that the 
15 sensor can monitor. In one embodiment, the sensor monitors the student's volitional 

behavior, such as his inputs into the computer, his facial expressions, his facial orientations 
and his eyes. In another embodiment, the sensor monitors the student's involuntary 
behavior, such as the sizes of his pupils. 

The controller analyzes one or more of the above behavior based on one or more 
20 rules. These rules are similar to the instructor's "intuition." For example, one rule is as 
follows: The student has lost concentration in the study materials if for a predetermined 
period of time, the student's inputs through a mouse have been in a window that does not 
contain study materials. Another rule is that if the student is not looking at the monitor 
showing the study materials for a predetermined period of time, the student has lost 
25 concentration in the study materials. From the analysis, the system provides an indication 
on the student's concentration level. 

Based on the indication, the system could react accordingly. Different reactions 
are applicable. Some examples include rewards, punishments, stimulation, and changing 
the study materials. 
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In another embodiment, due to the indication, the system asks the student a 
question, which can stimulate the student and can assess the student's understanding level 
in the study materials. From the student's response to the question, the system may 
change to more appropriate study materials and/or presentation style. 

5 The question-asking approach in the above embodiment does not have to be a 

reaction to the indication. In one embodiment, as the system is presenting study materials 
to the student, unexpected by the student, the system asks the student a question. After 
the student responds to the question, the system resumes back to present study materials 
to the student. In such an embodiment, the question tends to increase the concentration 

10 level of the student in the study materials. 

In yet another embodiment, the present invention also includes a calibrator, which 
calibrates the student's concentration-sensitive behavior before the behavior is being 
monitored to show concentration. One type of calibration establishes the student's 
behavior when the student is paying attention, and compares it with the student's behavior 

15 when the student is working on the study materials. Calibration typically improves the 
accuracy of the system. 

Other aspects and advantages of the present invention will become apparent from 
the following detailed description, which, when taken in conjunction with the 
accompanying drawings, illustrates by way of example the principles of the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



FIG. 1 shows one embodiment of the present invention. 
FIGS. 2A-B show one embodiment of a system implementing the present 
5 invention, 

FIG. 3 shows a set of steps to implement one embodiment of the present invention. 

FIG. 4 shows examples of volitional behavior monitored by the sensor in the 
present invention. 

FIG. 5 shows another embodiment of the present invention. 
10 Same numerals in FIGS. 1-5 are assigned to similar elements in all the figures. 

Embodiments of the invention are discussed below with reference to FIGS. 1-5. 
However, those skilled in the art will readily appreciate that the detailed description given 
herein with respect to these figures is for explanatory purposes as the invention extends 
beyond these limited embodiments. 
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DETAILED DESCRIPTION OF THE INVENTION 



FIG. 1 shows one embodiment of a computer-aided-educational system 100 of the 
present invention. As an overview of some of its components, the system 100 includes a 

5 selector 102, which selects study materials from a study-materials storage medium 108 to 
be presented to a student through a presenter 106. While the student is working on the 
study materials, a non-intrusive sensor 110 monitors the student's concentration-sensitive 
behavior, and sends its results back to a controller 104. Then the controller 104 based on 
one or more rules from a rules storage medium 1 12 analyzes the monitored results to 

10 provide through an indicator 1 14, an indication on the student's concentration level. In 
another embodiment, the selector is 102 also connected to the controller 104 to keep track 
of the study materials presented to the student. 

FIG. 2 A shows one embodiment of a system 150 implementing the present 
invention, preferably in software and hardware. The system 150 includes a server 

15 computer 152 and a number of client computers, such as 154, which can be a personal 
computer. Each client computer communicates to the server computer 152 through a 
dedicated communication link, or a computer network 156. 

FIG. 2B shows one embodiment of a client computer 154. It typically includes a 
bus 159 connecting a number of components, such as a processing unit 160, a main 

20 memory 162, an I/O controller 164, a peripheral controller 166, a graphics adapter 168 
and a network interface adapter 170. The I/O controller 164 is connected to components, 
such as a harddisk drive 172 and a floppy disk drive 174. The peripheral controller 166 is 
connected to peripheral components, such as a keyboard 176, a mouse 182, and a digital 
camera 180. The graphics adapter 168 is connected to a monitor 178; and the network 

25 interface adapter 170 is connected to the network 120, The network can be the Internet, 
an intranet, the world wide web and other forms of networks. 

Different components of the present invention can be in different elements shown 
in FIGS. 2A-B. For example, the presenter 106 and the sensor 110 can be in a client 
computer; the selector 102, the controller 104, the study-materials storage medium 108, 

30 the rules storage medium 1 12 and the indicator 1 14 can be in the server computer 152. In 
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another embodiment, the selector 102, the controller 104 and the indicator 1 14 are also in 
a client computer. Different components can be in different elements in the above 
description. Nonetheless, there is no restriction preventing all components to reside in one 
element, such as a client computer. A number of operations in the present invention can 
5 be implemented by software, which is controlled, for example, by the processing unit 160. 
In yet another embodiment, the number of operations implemented by software can be 
stored in a storage-medium, which can be, for example, the main memory 162 or a CD 
read-only-memory. 

The present invention is applicable to teach any subject or materials that can be 
10 taught by a computer. The teaching period may last one semester or a year, or just one 

class session. The materials may cover inter-disciplinary areas, such as electrical 

engineering and thermodynamics, or computer networking and programming techniques. 

The materials may just be for training a field engineer on a new product. In the following, 

mathematics is the subject used to illustrate the present invention. 
15 In one embodiment, the subject is divided into major-topics, with each major-topic 

subdivided into minor-topics, and with each minor-topic further subdivided into line-items. 

Each line-item typically covers one well-defined area in the subject. In another 

embodiment, the subject is further divided into more levels below the line-items; and in a 

third embodiment, the subject is just divided into line-items. 
20 As an example of line-items, if the major-topic is high school algebra, then it can 

be divided into the following line-items, with bracketed terms served as comments: 

High School Algebra (the major-topic) 
(Minor-topics under the major-topic) 
25 Decimal Numbers 

Polynomials 
Linear Equations 
Quadratic Equations 

30 Integers 
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(Line-items under the minor-topic of integers) 
Addition & Subtraction (Difficulty level 1) 
Multiplication (Difficulty level 2) 
Division (Difficulty level 2) 
Prime Numbers (Difficulty level 3) 
Factorization (Difficulty level 3) 
Common Divisor (Difficulty level 4) 

Fractions 

(Line-items under the minor-topic of fractions) 
Addition & Subtraction (+/-) with Common Denominator 

(Difficulty level 3) 
+/- with Integers (Difficulty level 4) 
+/- without Common Denominator (Difficulty level 5) 
Multiplication and Divisions (*/) with Integers (Difficulty level 5) 
V with fraction (Difficulty level 6) 
Compound Fractions (Difficulty level 6) 
Fraction Reduction (Difficulty level 7) 
Ratios and Proportions (Difficulty level 7) 

Another example with the minor topic being differential calculus is as follows 

Calculus (major topic) 

Differential calculus (minor topic) 
Fractions (Difficulty level 1) 
Polynomials (Difficulty level 1) 
Exponential Functions (Difficulty level 1) 
Differentiation (Difficulty level 2) 
Differentiate a sum (Difficulty level 3) 
Differentiate a product (Difficulty level 3) 



Differentiate a quotient (Difficulty level 4) 
In one embodiment, each line-item has a difficulty level. The bracketed difficulty level 
next to each line-item in the above example indicates how difficult one line-item is relative 
to other line-items in the subject, or how significant one is relative to another. A line-item 
5 with a low difficulty level is a relatively easy line-item or a relatively less important line- 
item. Typically, a student learning a subject starts from learning line-items at the lowest 
difficulty level. 

The lists of items in the above examples are generated based on expert knowledge 
on the subject of mathematics. With the proper instruction, such as through reading the 
10 present specification, generating such lists with the difficulty levels should be obvious to 
experts in the subject. The more knowledgeable the expert, the more complete the sets of 
items. 

In one embodiment, each line-item is represented by a line-item root, which 
includes the line item and its root. In the above example, the root of a line-item includes 

15 its subject, its major topic and minor topic. 

In one embodiment, the selector 102 starts the learning process by selecting a line- 
item with the lowest difficulty level If there are a number of those, one of them is 
randomly selected. Study materials for that line-item are retrieved from the study- 
materials storage medium 108 to be presented to the student. After presentation, the 

20 selector 102 selects another line-item with the lowest difficulty level among all the un- 
selected line-items, and the process repeats. For this embodiment, each line-item also 
includes a mode attribute, which is changed from the un-selected to the selected mode 
after the study materials for that line-item has been selected to be worked on by the 
student, 

25 To select a set of study materials from the study-material storage medium 108, the 

selector 102 sends the line-item root to the storage medium 108 to retrieve the 
corresponding study materials. Typically, there are a number of sets of study materials in 
the storage-medium 108, and they can be in the following format: 
(line-item root, mode, study materials) 

30 The following serves as examples of study materials for differentiating polynomial: 
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First, the system teaches the approach to generate derivatives based on the 
basic principle in differentiation, such as: 

df(x)/dx - lim h ->o ((f(x+h) - f(x))/h) 
Then the system teaches the generalized equation, such as: 
5 ((dEaixVdx) - (Si * aiX M ) 

Finally, the system teaches the importance of and the way to find optima 
and minima by solving the following equation: 
((dEaixVdx) = 0 

Based on the line-item root, and with one set of study materials per line-item, the selector 
10 102 retrieves from the study-materials storage-medium 108, the corresponding set of 

study materials. Creating study materials on a subject should be obvious to experts in the 

subject, and will not be further discussed in this application. 

The selector 102 then sends the retrieved study materials to the presenter 106. 

The study materials can be a document with no questions, arranged as a list of screens. 
15 The presenter 106 typically includes the monitor 178, which presents the study materials 

to the student, who can go from one screen to another with the keyboard 176, or the 

mouse 182. In another embodiment, the study materials are broadcast through a radio. 

As the student is working on the study materials presented through the radio, the student's 

concentration-sensitive behavior is monitored automatically. 
20 In another embodiment, the study materials only have questions. Typically, 

students gain a better understanding on a subject through actively working on questions 

than through passively reading study materials. In one embodiment, each question is 

embedded in a question entry, which is of the following format: 
(line-item root, mode, question-body, answer). 
■ 25 The term "question-body" describes the body of a question. The following serves as an 

example: 



Subject: Mathematics, 
Major-topic: High School Algebra, 
30 Minor-topic: Fraction. 
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Line-item: +/- with common denominator 
Mode: Un-selected 



Answer Question-body 
5 28/37 What is the sum of 2/37, 3/37, 8/37 and 15/37? 

-2/43 17/43 -25/43 +6/43 =? 

The selector 102 sends to the study-materials storage medium 108 the line-item root to 
retrieve the set of questions with the same line-item root. 

An example of study materials with questions are for the line-item of differentiating 
10 exponential functions. A number of questions are generated, including the question on 
expanding an exponential function based on Taylor expansion, the question on 
differentiating the Taylor-expanded exponential function, whose answer is the original 
Taylor-expanded exponential function, and the question on differentiating the exponential 
function, whose answer is the exponential function. 
15 In another embodiment, the study materials include study materials with questions 

and study materials without questions. 

Note that the formats of the study materials may change as the student progresses. 
The student can learn one line-item based on questions, and another based on study 
materials with no questions. As an example, for differential calculus, of the different line- 
20 items, all of them can be learnt through either study materials with or without questions, 
except for the line-item of differentiation, which is typically learnt without questions. That 
study-materials cover the general differentiation concept, such as the following: 

df(x)/dx = lim h -* 0 ((f(x+h) - f(x))/h) 
FIG. 3 shows a set of steps 250 to implement one embodiment of the present 
25 invention. First, the presenter 106 presents (step 252) the selected study materials to the 
student. As the student is working on the study materials, the sensor 1 10 monitors (step 
254) more than once the student's concentration-sensitive behavior, and feeds those 
monitored results to the controller. The controller 104 analyzes (step 256) the results 
based on one or more rules to provide (step 258) an indication on the student's 
30 concentration. Based on the indication, the system reacts (step 260) accordingly. 
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A type of concentration-sensitive behavior is a type of behavior that is sensitive to 
one's concentration. As one's concentration changes, such a type of behavior changes 
accordingly. The behavior can be physical, psychological, biological, emotional and 
physiological. 

5 In the step of monitoring (step 254), the sensor automatically monitors more than 

once the student's concentration-sensitive behavior while the student is working on the 
study materials. Instead of just monitoring once to determine concentration level, 
monitoring more than once increases the accuracy in determining the student's 
concentration level. For example, a student is concentrating on the study materials. A 

10 mosquito lands on the back of his right hand. As the student is trying to hit the mosquito, 
the system monitors him. The indication based on that image alone is a correct indication 
of the student's concentration level in the study materials at that specific instant. 
However, the single measurement is not a good indication of the student's actual 
concentration level in the study materials-that single measurement is an outlying point 

15 that should be deleted. Instead of just one single result, this embodiment monitors more 
than once the student's behavior, which enhances identifying a pattern to eliminate 
outlying points. 

The monitoring step does not have to stop after monitoring twice. The monitoring 
step can continue in a periodic manner, such as once every two seconds. In the 

20 embodiments of monitoring more than once or monitoring periodically, the results can be 
analyzed to identify patterns. 

The behavior monitored more than once can be of the same type, or can be of 
different types. In one embodiment, in monitoring more than once, the sensor monitors 
the same type of behavior each time. In another embodiment, in monitoring more than 

25 once, the sensor monitors more than one type of behavior; for example, a first monitoring 
process is on one type of behavior, and a second monitoring process is on another type of 
behavior. In monitoring more than one type of behavior, the sensor may include more 
than one type of sensor, which can monitor more than one type of behavior substantially 
simultaneously. Monitoring more than one type of behavior is similar to monitoring one 
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type of behavior more than once, in the sense that both approaches increase the accuracy 
in determining the student's concentration level. 

FIG. 4 shows examples of different types of concentration- sensitive behavior, 
which are volitional 300. In one embodiment, the sensor 1 10 monitors the student's 

5 volitional inputs entered into the computer (box 302). One type of volitional inputs is 
entered through the keyboard 176 or the mouse 182. The study materials can be 
presented to the student through the monitor 178. As the student works on the study 
materials, he enters commands through the keyboard 176 or a position-pointing device, 
such as the mouse 182, or arrow buttons of the keyboard 176. The inputs may be the 

10 downward or upward arrows on the keyboard or the mouse, or may be the typing speed 
through the keyboard. 

In one embodiment, the sensor 110 monitors the speed of inputs by the student as 
a function of time. There are different ways to monitor the speed of inputs, such as 
polling periodically the corresponding devices of those inputs. Such monitoring process 

15 should be obvious to those skilled in the art, and will not be further described. 

As the student starts working on the study materials, the inputs are entered at a 
certain speed. As the student gets tired, or as the student loses concentration, this speed 
typically decreases. In this embodiment, the student's input speed is compared with a 
reference speed to identify changes. 

20 There are a number of methods to determine the reference speed. In one 

embodiment, this reference speed is set through randomly sampling many students. Based 
on the students' responses on similar study materials, a reference speed is determined. In 
another embodiment, the student's initial speed becomes the reference speed. This initial 
speed may be found for example by averaging the student's speed across five minutes, 

25 such as from the first one minute to the first six minutes of the student's usage. 

Different types of study materials typically have different reference speeds. For 
example, if the study materials include no pictures, the input speed may be slow because 
the student has to read an entire screen of text. If the student has to compose a sentence, 
the speed is likely to have frequent short pauses because the student has to think to 

30 compose the sentence. To accommodate such variations, in one embodiment, the 
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reference speed is a function of the difficulty level of the study materials. As the student 
progresses in working on the study materials, the difficulty level of the study materials 
typically increases. In one embodiment, the reference speed is divided by the following 
factor: 

5 (The difficulty level of the study material * a constant). 

With the above equation, as the difficulty level increases, the reference speed decreases 
accordingly. In this embodiment, the reference speed tracks the difficulty level of the 
study materials. 

As discussed above, by monitoring the speed of the student's inputs, the system 
10 100 can provide an indication of the student's concentration. Thus, one rule is as follows: 
If the speed of the student's volitional inputs across a predetermined period of time 
is significantly lower than the reference speed, the student has lost concentration in 
the study materials. 

In one embodiment, the predetermined period of time is two minutes; and more than three 

15 times slower is considered as significantly lower. 

In another embodiment, the study materials are presented through a multi-windows 
environment. The student enters inputs into the system, such as through a position- 
pointing device, like the mouse 182, or through the keyboard 176. In one embodiment, 
the sensor 110 in this embodiment is implemented through software, which periodically, 

20 such as every two seconds, polls the operating system or the device drivers of the 

position-pointing device. The polling determines if there have been any inputs. Writing 
such software to monitor such inputs to the system should be obvious to those skilled in 
the art, and will not be further described in this application. In such an embodiment, one 
rule is as follows: 

25 If for a predetermined period of time, the inputs have been entered outside the 

window where the study materials reside, the student has lost concentration in the 
study materials. 

In yet another embodiment, the study materials are presented in a window 
environment that has a focus window, and the sensor 110 can sense the focus window, for 
30 example as in the above embodiment. In such an embodiment, one rule is as follows: 
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If the study materials are not in the focus window for a predetermined period of 
time, the student has lost concentration in the study materials. 
In one embodiment, the predetermined amount of time is more than one minute. If 
monitoring is performed every three seconds, in one minute, the system would have 

5 performed 20 measurements. 

In yet another embodiment, the sensor 1 10 senses another type of volitional 
behavior, which is based on the student's face. In this embodiment, the monitor 178 
presents study materials. The sensor 110 including the digital camera 180 are positioned 
adjacent to the monitor 178, as shown, for example in FIG. 2 A. With the camera 

10 positioned where it is, when the student is looking at the monitor to work on the study 
materials, the digital camera 180 could take digital images of the student's face. Taking 
the digital images to generate numerous bits of data should be obvious to those skilled in 
the art and will not be further described in this application. 

To improve the performance of this embodiment, before the step of monitoring, 

15 the present invention includes the step of calibration through imaging. One calibration 
technique enters the student's image before the student works on the study materials, and 
uses that image as the reference to compare with other images. For example, before the 
student starts working on the study materials, he is asked to look at the monitor 178 with 
a message box having a message such as "LOOK AT ME," and with a picture of two eyes 

20 staring at the student. Then, the digital camera 180 takes a reference image of the 
student's face, who typically looks at the two eyes. 

The reference image should be analyzed. That image includes not only the 
student's face, but also background information, such as the wall of a room. In one 
embodiment, the controller assumes that the student's two eyes are looking at the two 

25 eyes in the monitor. Based on this assumption, the student's face is determined. Such 
image recognition techniques are disclosed for example in "Computer Recognition of 
Human Faces," written by Takeo Kanada, and published by Birkhauser Verlag, Basel and 
Stuttgart, in 1977. Even if the distance between the monitor and the student's face 
increases, the relative distances among different features on his face remain the same. In 

30 one embodiment, the monitoring step focuses on relative distances to re-calibrate the 
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student's face. Such image recognition techniques should be obvious to those skilled in 
the art, and will not be further described in the present application. 

One type of facial information is the facial orientation (box 304). The controller 
104 connected to the digital camera 180 calibrates the facial orientation when the student 
5 is looking at the monitor 178. This reference image could be just the oblong shape of the 
face. After calibration, when the student starts working on the study materials, the digital 
camera 180 regularly captures the facial image, such as once every few seconds. All 
information in that image is removed leaving behind the orientation of the face. These 
orientations are compared with the reference image to check for differences. The distance 
10 between the monitor and the student's face may change. To compensate for such 

changes, in one embodiment, the controller uses the ratio of the longest horizontal to the 
longest vertical distance of the oblong shape. If the captured facial orientation is 
significantly different from the reference facial orientation, the student is not looking at the 
monitor. The student may be looking away from the computer or drooping while falling 
15 asleep. In such an embodiment, one rule is as follows: 

If the student's facial orientation is significantly different from its reference image 

as shown in two consecutive monitoring processes, the student has lost 

concentration in the study materials. 
In one embodiment, two images are considered significantly different if their horizontal-to- 
20 vertical-distance ratios differ by more than 20%. 

Another type of facial information is the condition of the eyes (box 306). If the 
eyelids are covering significant portions of the irises, the student's eyes are closing. In 
such an embodiment, one rule is as follows: 

If the eyelids cover more than 60% of the irises as shown in two consecutive 
25 monitoring processes, the student has lost concentration in the study materials. 

Another type of facial information is the student's facial expressions (box 308), 
such as whether the student is frowning or not. In such an embodiment, one rule is as 
follows: 

If the student frowns in two consecutive monitoring processes, the student is 
30 concentrating on the study materials. 
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Concentration-sensitive behavior can be involuntary. In one embodiment, the 
sensor 1 10 monitors the sizes of the student's pupils, assuming that a student's pupil 
dilates if the student loses focus and concentration. In such an embodiment, one rule is as 
follows: 

5 If the average size of the student's pupils dilates by more than 20% as compared to 

the average size of the reference image for a predetermined amount of time, the 
student has lost concentration in the study materials. 
Other examples of involuntary concentration-sensitive behavior include the 
student's heart beat, breathing rate, body temperature and whether the student's sweat has 
10 increased. With appropriate sensors and rules, these involuntary behavior can be 
monitored to provide indications on whether the student has lost concentration in the 
study materials. 

There are many other types of concentration-sensitive behavior. Different types of 
behavior coupled with their corresponding sensors and rules should be able to indicate the 

15 student's concentration level. 

More than one type of the student's concentration-sensitive behavior can be 
monitored by one or more sensors. In fact, one type can be volitional, with the other type 
involuntary. Including different types of behavior tends to increase the accuracy of 
identifying the student's concentration level. With more than one type of behavior being 

20 monitored, the system may not have to monitor each type of behavior more than once to 
identify the student's concentration level. An example of a rule for such an embodiment is 
as follows: 

If the student's facial orientation is different from the reference image by more 
than 20% while the student's eyelids are covering more than 60% of the irises, the 
25 student has lost concentration in the study materials. 

Such rules should be obvious to experts in the field of human perception, and will not be 
further described in this application. 

The above embodiments describe whether the student has lost concentration or 
not. However, the invention is also applicable to indicate the student's degree of 
30 concentration, such as ranging from low, medium to high. For example, if the student has 
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not lost concentration in the study materials for a long period of time, the student's 
concentration level is high. Another example is that if the student's eyes are wide open 
with his inputs through the mouse moving down the study materials in a fairly constant 
speed for a long duration of time, such as five minutes, the student's concentration level is 
5 also high. 

In another embodiment, if the controller 104 decides that the student has not lost 
concentration for a long period of time, such as ten minutes, the controller 104 averages 
the captured results during that time frame-with outlying points removed-and treats the 
averaged results as the reference, which will be used to compare with subsequent captured 

10 results, to determine if the student has lost concentration in the study materials. The 
reference can be a reference image, such as the student's face, or the student's input 
speed, as appropriately modified by the study materials' difficulty level, or other 
monitored results. As the student continues working on study materials, this reference can 
be updated regularly by averaging it with subsequent captured results, which also show 

15 that the student has not lost concentration. Unlike many of the previously described 

references, which are static, this type of reference is typically not a constant, and is known 
as a dynamic reference. It is usually more closely tailored to the student. With more data 
used to generate the dynamic reference, its accuracy is typically better than the static 
references. 

20 In yet another embodiment, the system 100 asks for the student's identity, such as 

the student's name, when the student starts working on the study materials. After the 
student enters his identity, it is stored in the system 100. The student's reference 
information, whether static or dynamic, is stored with the student's identity in the memory 
of the system 100, such as its harddisk. After the first working session, if the student 

25 wants to work on study materials through the system 100 again, the system retrieves from 
its memory the student's reference information. For such an embodiment, the retrieved 
reference information can replace the step of calibration. If the reference is of the dynamic 
type, the retrieved information is regularly updated. 

Based on one or more of the above concentration-sensitive behavior coupled to 

30 one or more appropriate rules, the indicator 114 provides an indication on the student's 
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concentration (step 258), or the student's degree of concentration. Such indication can be 
as simple as changing the state of a register--a high logic level indicates the student has not 
lost concentration, while a low logic level indicates the student has. Another indication 
can be printing a report indicating that the student's degree of concentration in the study 
5 materials for a period of time. 

In one embodiment, the system reacts according to the indication (step 260). 
Some examples of reactions include stimulation, rewards, punishments or changing the 
study materials. 

If the indication is that the student has lost concentration in the study materials, 
10 one way the controller 104 can help the student to re-focus on the study materials is 

through stimulation. This includes presenting a real-life application of the study materials 
that the student has lost concentration in. The stimulation can be through sound. It can 
be visual effects, including changing the screen temporarily and then restoring to the 
previous screen. 

15 Another type of stimulation includes allowing the student to play a game. This 

stimulation is applicable if the student has been working on the study materials for a long 
duration of time, and should have a break. Thus, after the student has worked for a long 
period of time, such as 45 minutes, and is losing concentration in the study materials, the 
controller can pose the student a question, such as, "Do you want to take a break and play 

20 a game?" If the student wants to, in one embodiment, the controller accesses a game from 
the study-materials storage medium, which includes a number of games. The game serves 
as a diversion. Not only does it distract the student's mind for some time, the game also 
relaxes and entertains the student. After the game, presentation is resumed on the study 
materials. 

25^ Another form of reaction is a reward. If the student has been concentrating for a 

long period of time, at the end of a section in the study materials, the system reacts by 
praising the student audibly through a speaker, or visually through the monitor with words 
like "TIME FOR A SNACK!" Other examples of rewards include playing a short piece of 
music, presenting a joke, a factoid on an interesting subject, or playing a short animation 

30 or video clip. 
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A further form of reaction is punishment. This includes generating a report 
indicating that the student has lost concentration for a long period of time so that the 
student's supervisor can punish the student accordingly. Another punishment may be an 
audible reprimand, such as "PAY ATTENTION!" 

5 The system can also change the study materials according to the monitored results. 

If the student has lost concentration in working on the study materials for a predetermined 
amount of time, the system can react by changing the study materials to a different set of 
materials. Also, the presenter 106 may change the presentation style accordingly, such as 
by reducing the speed of presentation through increasing the line spacing of the text or the 

10 size of the image to present to the student. 

In yet another embodiment, due to the indication, the system asks the student a 
question. This question can stimulate the student, and help the student to re-focus in the 
study materials. Typically, the question is based on the study materials just presented to 
the student. 

15 As a side note, while the controller 104 through the sensor 1 10 monitors the 

student's concentration-sensitive behavior, the controller 104 can also track the 
corresponding study materials being presented to the student. Such an embodiment has 
the added benefit of tying the indication with the corresponding study materials presented 
to the student. 

20 Back to the embodiment that asks the student a question, this embodiment can be 

achieved, for example, through the selector 102 sending to the study-materials storage 
medium 108 the line-item root of the study materials just presented. From the line-item 
root, a set of questions with the same line-item root is retrieved, and one of those question 
is randomly selected for the student. Other ways may be used to generate a question on 

25 materials just presented to the student. One simple way is to randomly select a sentence 
that has just been presented to the student, and change the syntax of that sentence into a 
question. 

The embodiment on asking questions has a number of benefits. Even if the student 
does not know the answer to the question, typically, the student is stimulated by the 
30 question. Also, the question can be used to assess the student's understanding level on the 
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materials just presented to the student. After the student answers the question, if the 
answer is correct, the controller 104 can praise the student appropriately. If the answer is 
not correct, the student may not understand what has just been presented. The controller 
104 has a number of options. For example, the study materials just presented can be 

5 presented to the student again; the location as to where he can find the answer to the 
question can be presented to the student; the location of the answer can be hyperlinked to 
the location of the wrong answer if the student activates an icon shown on the presenter 
106; the presenter 106 presents study materials that are easier than the one just presented 
to the student, such as one with a lower difficulty level; or the presenter 106 can resume 

10 presenting, and ignore the wrong answer altogether. 

The above embodiment on asking questions can be modified to focus on increasing 
the student's concentration level, FIG. 5 shows such an embodiment 350. The system 
presents (step 352) study materials to the student. Then, the system asks the student a 
question unexpectedly (step 354). As an example, if the study material is presented 

15 through the monitor, unexpectedly, the entire screen changes. From a screen of study 
materials, the system suddenly changes the screen to display a question. The unexpected 
nature of the change, together with the displaying of the question stimulate the student. 
To further enhance the effect of stimulation, the system can spell out the question while 
displaying it. Typically the question is based on the study materials the student has been 

20 working on. After the student responds to the question, the system resumes (step 356) 
presenting the study materials to the student. 

The question stimulates. Right before the question is presented, the student may 
be concentrating or may not be concentrating. Either way, the student, unlikely to be 
aware that a question is coming, is suddenly confronted with a question. Independent of 

25 whether the student knows the answer, the question typically increases the student's 
concentration level. Also, responding to a question is an active learning approach, as 
compared to the passive learning approach of reading. The more active learning approach 
together with the unexpected nature of the question tend to increase the student's memory 
retention in the subject matter covered by the question, 
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Another benefit provided by the question is that the student's answer to the 
question provides an indication on the student's understanding level in the study materials. 
As described above, if the answer is wrong, the system can go over that part of the study 
materials, or can reduce the difficulty levels of the study materials to be presented to the 

5 student. In another embodiment, the question is just for increasing concentration; the 
system ignores the answer, and continues on with the presentation. 

The student might have stopped working on the study materials altogether. For 
the embodiment that monitors the student's inputs, if there is no inputs for a 
predetermined amount of time, such as ten minutes, the system assumes that the student 

10 has totally stopped working on the study materials. 

The present invention teaches sensing through different types of non-intrusive 
sensors, which are defined as sensors that do not cause the student physical pain and 
suffering when they are sensing the student's concentration-sensitive behavior. As 
technology progresses, sensors that are intrusive today can become non-intrusive in the 

15 future, for example, sensors that monitor the student's brain waves, which can be a type of 
concentration-sensitive behavior. 

Rules are stored in the rules storage medium. However, in one embodiment, the 
rules have previously been embedded in the software implementing the present invention. 
With rules already embedded in the software, there is no need for accessing the rules, and 

20 there is no need for the system to have the rules storage medium. 

In the above embodiments, the student's behavior is monitored more than once 
before the step of analysis. In another embodiment, the monitoring step and the analysis 
step are intermixed. Instead of monitoring more than once and then analyzing the results, 
in this embodiment, the sensor monitors one type of behavior, with the result analyzed. 

25 Then the sensor monitors the same or a different type of behavior, with the result 
analyzed. 

In yet another embodiment, the steps in the present invention repeat. For example, 
after the step of reacting according to the indication (step 260) or providing an indication 
(step 258), the invention repeats from the step of monitoring automatically (step 254). In 
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this embodiment, study materials are continually presented to the student, although the 
study materials might be changed due to the reaction (step 260). 

The rules discussed can be self-adapting. In other words, the controller 104 can 
change a rule after applying the rule to a number of situations and after analyzing the 
results. This can be done, for example, in a fossy-logic system. 

Other embodiments of the invention will be apparent to those skilled in the art 
from a consideration of this specification or practice of the invention disclosed herein. It 
is intended that the specification and examples be considered as exemplary only, with the 
true scope and spirit of the invention being indicated by the following claims. 
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CLAIMS 



We claim; 



1 1 . A computer-aided-educational method for educating a student on a subject 

2 through a computer, which includes a presenter and a non-intrusive sensor, the method 

3 comprising the steps of: 

4 presenting study materials on the subject to the student through the presenter; 

5 monitoring automatically more than once the student's concentration-sensitive 

6 behavior through the sensor while the student is studying the materials; 

7 analyzing the monitored results based on one or more rules; and 

8 providing an indication on the student's concentration level based on the analysis. 

1 2. A computer-aided-educational method as recited in claim 1 wherein for the step of 

2 monitoring, more than one type of the student's concentration-sensitive behavior is 

3 monitored, 

1 3. A computer-aided-educational method as recited in claim 1 wherein for the step of 

2 monitoring, the behavior monitored is of the same type, 

14. A computer-aided-educational method as recited in claim 1 wherein the student's 
2 behavior is volitional, 

3 

15. A computer-aided-educational method as recited in claim 4 wherein the type of 
2 behavior includes the student's volitional inputs, 

16. A computer-aided-educational method as recited in claim 5 wherein: 
2 the sensor monitors the speed of the student's inputs; and 
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3 one rule is that if the speed of the student's volitional inputs across a 

4 predetermined period of time is significantly lower than a reference speed, the student has 

5 lost concentration in the study materials. 

1 7. A computer-aided-educational method as recited in claim 6 wherein whether a 

2 decrease in speed is significant or not depends on the difficulty level of the study materials. 

18. A computer-aided-educational method as recited in claim 5 wherein: 

2 the study materials are presented in one or more windows of a multi-window 

3 environment; 

4 the study materials reside in at least one of those windows; and 

5 one rule is that if, for a predetermined amount of time, the student's inputs are not 

6 in the one or more windows displaying the study material, the student has lost 

7 concentration in the study materials. 

1 9. A computer-aided-educational method as recited in claim 5 wherein: 

2 the study materials are presented in a window environment with a focus window; 

3 and 

4 one rule is that if the study materials are not in the focus window for a 

5 predetermined amount of time, the student has lost concentration in the study materials. 

1 10. A computer-aided-educational method as recited in claim 4 wherein the volitional 

2 behavior is based on the student's face. 

1 11. A computer-aided-educational method as recited in claim 10 wherein before the 

2 step of monitoring, the method comprises the step of calibrating the student's face to 

3 generate a reference image. 

1 12. A computer-aided-educational method as recited in claim 10 wherein the behavior 

2 includes the student's facial orientation. 
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1 13. A computer-aided-educational method as recited in claim 1 1 wherein one rule is 

2 that if results from two consecutive monitoring of the student's face are different from the 

3 reference image, the student has lost concentration in the study materials. 

1 14. A computer-aided-educational method as recited in claim 10 wherein the behavior 

2 includes the condition of the student's eyes. 

1 15. A computer-aided-educational method as recited in claim 10 wherein the behavior 

2 includes the student's facial expression. 

1 16. A computer-aided-educational method as recited in claim 1 1 wherein: 

2 the sensor includes a camera; 

3 the presenter includes a monitor; and 

4 the step of calibrating includes asking the student to look at the monitor. 

1 17, A computer-aided-educational method as recited in claim 1 wherein the student's 

2 behavior monitored is involuntary. 

1 18. A computer-aided-educational method as recited in claim 17 wherein the behavior 

2 includes the size of the student's pupils. 

1 19. A computer-aided-educational method as recited in claim 1 further comprising the 

2 step of reacting according to the indication provided. 

1 20. A computer-aided-educational method as recited in claim 19 wherein the reaction 

2 includes punishing the student for losing concentration in working on the study materials. 
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1 21. A computer-aided-educational method as recited in claim 19 wherein the reaction 

2 includes stimulating the student to re-focus on the study materials if the student has lost 

3 concentration in the study materials. 

1 22. A computer-aided-educational method as recited in claim 21 wherein if the student 

2 has worked on the study materials for a pre-determined amount of time, the reaction 

3 includes allowing a student to play a game and then resuming back to the study materials. 

1 23. A computer-aided-educational method as recited in claim 19 wherein the reaction 

2 includes changing to a different study materials to present to the student. 

1 24. A computer-aided-educational method as recited in claim 19 wherein the reaction 

2 includes asking the student a question. 

1 25. A computer-aided-educational method as recited in claim 24 wherein the study 

2 materials to be presented to the student depends on the student's response to the question. 

1 26. A computer-aided-educational method as recited in claim 24 wherein the speed of 

2 presenting the study materials to the student depends on the student's response to the 

3 question. 

1 27. A computer-aided-educational method for educating a student on a subject 

2 through a computer, which includes a presenter, the method comprising the steps of: 

3 presenting study materials on the subject to the student through the presenter; 

4 asking by the computer, the student a question, when the student is unlikely to 

5 expect a question; and 

6 resuming presenting study materials on the subject to the student through the 

7 presenter; 

8 such that the unexpected nature of the question stimulates the student to enhance 

9 the student's concentration level in the study materials. 
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1 28. A computer-aided-educational method as recited in claim 27 wherein the student's 

2 answer to the question does not affect the study materials resumed to be presented to the 

3 student. 

1 29. A computer-aided-educational method as recited in claim 27 wherein: 

2 the student's answer to the question provides an indication on the student's 

3 understanding level in the study materials presented; and 

4 the difficulty level of the study materials resumed to be presented depends on the 

5 indication. 

1 30. A computer-aided-educational method as recited in claim 1 wherein for the step of 

2 monitoring, between two consecutive monitoring of the student's concentration-sensitive 

3 behavior, the method further comprises the step of analyzing the monitored result based 

4 on one or more rules, 

1 31. A computer-aided-education method as recited in claim 1 : 

2 further comprising the step of determining a reference for the student; and 

3 wherein the step of analyzing includes the step of comparing the student's 

4 monitored results with the reference to identify the student's concentration. 

1 32. A computer-aided-education method as recited in claim 3 1 wherein the reference is 

2 a dynamic reference, which incorporates monitored results indicating the student has not 

3 lost concentration in the study materials. 

1 33. A computer-aided-educational method as recited in claim 3 1 further comprising 

2 the steps of: 

3 establishing the student's identity; and 

4 storing the identity with the reference to be used by the student in the future. 
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1 34. A computer-aided-educational system for educating a student on a subject 

2 comprising: 

3 a presenter for presenting study materials on the subject to the student; 

4 a non-intrusive sensor for monitoring more than once the student's concentration- 

5 sensitive behavior while the student is studying the materials; 

6 a controller for analyzing the monitored results based on one or more rules; and 

7 an indicator for providing an indication on the student's concentration level based 

8 on the analysis. 

1 35. A computer-aided-educational system as recited in claim 34 wherein the student's 

2 concentration-sensitive behavior is volitional 

1 36. A computer-aided-educational system as recited in claim 34 wherein the student' s 

2 behavior is involuntary. 

1 37. A computer-aided-educational system as recited in claim 34 wherein the system 

2 reacts according to the indication provided by the indicator. 
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ABSTRACT 



A computer-aided-educational method and system that consider a student's 
concentration level when teaching the student. The system monitors automatically more 

5 than once the student's concentration-sensitive behavior while the student is working on 
the study materials. Through monitoring the student's volitional or involuntary behavior, 
the system provides an indication on the student's concentration level. Based on the 
indication, the system could react accordingly, Reactions include, for example, providing 
rewards, punishments, and stimulation; or changing the study materials. The system can 

10 also react by asking the student a question to stimulate the student or to assess the 

student's understanding level. Based on the student's response, the system may change to 
more appropriate study materials, or presentation style. 
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^eterP.Tong 

1 807 Lfroetree Lane, Mountain View,CA 94040 



Direct Telephone Calls To: 
Peter P, Tong 
(415) 961-7789 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made on 
information and belief are believed to be true; and further that these statements were made with the knowledge that 
willfulfalse statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 
of Title 18 of the United States Code and that such willfulfalse statements may jeopardize the validity of the 
application or any patent issued thereon. 

Full Name of Inventor: Chi Fai Ho Citizenship: China 

Residence/Post Qffice Address: 4816 Cabello Court, Union City, CA 94587 . 



Inven^fH^ignaturj 



Date 



i 



(Use Page Two For Additional Inventors) Signature(s)) 
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DECLARATION AND POWER OF ATTORNEY 
FOR PATENT APPLICATION (continued) 



ATTORNEY DOCKET NO. 



110 



Full Name of #2 joint inventor: 
Residence/Post Office Address: 



Inventor's Signature 



Full Name of #3 joint inventor: 
Residence/Post Office Address: 



Peter P. Tong 



1807 Limetree Lane 



Citizenship: USA 



Mountain View, CA 94040 



Date 



Citizenship: 



Inventor's Signature 

Full Name of #4 joint inventor: 
Residence/Post Office Address: 



Inventor's Signature 

Full Name of #5 joint inventor: 
Residence/Post Office Address: 



Date 



Citizenship: 



Date 



Citizenship: 



Inventor's Signature 



Date 



Full Name of #6 joint inventor: 
Residence/Post Office Address: 



Citizenship: 



Inventor's Signature 



Date 



Full Name of #7 joint inventor: 
Residence/Post Office Address: 



Citizenship: 



Inventor's Signature 



Date 



Full Name of #8 joint inventor: 
Residence/Post Office Address: 



Citizenship: 



Inventor's Signature Date 

(Use Next Page For Additional Inventors) Signature(s)) 
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